Ombining S Elf - a Ttention and C Onvolution
نویسندگان
چکیده
Current end-to-end machine reading and question answering (Q&A) models are primarily based on recurrent neural networks (RNNs) with attention. Despite their success, these models are often slow for both training and inference due to the sequential nature of RNNs. We propose a new Q&A architecture that does not require recurrent networks: Its encoder consists exclusively of convolution and self-attention, where convolution models local interactions and self-attention models global interactions. On the SQuAD dataset, our model is 3x to 13x faster in training and 4x to 9x faster in inference, while achieving equivalent accuracy to recurrent models. The speed-up gain allows us to train the model with much more data. We hence combine our model with data generated by backtranslation from a neural machine translation model. On the SQuAD dataset, our single model, trained with augmented data, achieves 84.6 F1 score on the test set, which is significantly better than the best published F1 score of 81.8.
منابع مشابه
F Ast and a Ccurate R Eading C Omprehension by C Ombining S Elf - a Ttention and C Onvolution
Current end-to-end machine reading and question answering (Q&A) models are primarily based on recurrent neural networks (RNNs) with attention. Despite their success, these models are often slow for both training and inference due to the sequential nature of RNNs. We propose a new Q&A architecture that does not require recurrent networks: Its encoder consists exclusively of convolution and self-...
متن کاملتأثیر امواج الکترومغناطیس با فرکانس بسیار پایین بر درد حاد و مزمن در موشهای سوری
Background and Objective: Use of extremely low frequency magnetic field) ELF-MF (has been reported in increasing blood sugar, cholesterol, triglyceride and reduction of withdrawal syndrome signs of morphine. Since pain is one of the main considerations and usually analgesic drugs are not very useful and have side effects, therefore, the present project was carried out using formalin test to eva...
متن کاملELF Radiation Produced by Electrical Currents in Sprites
Measurements of ELF-radiating currents associated with sprite-producing lightning discharges exhibit a second current peak simultaneous in time with sprite luminosity, suggesting that the observed ELF radiation is produced by intense electrical currents flowing in the body of the sprite.
متن کاملObservations of amplitude saturation in ELF/VLF wave generation by modulated HF heating of the auroral electrojet
[1] We present detailed observations of the onset of amplitude saturation in ELF/VLF waves generated via modulated HF heating of naturally-forming, large-scale current systems, such as the auroral electrojet. Broadband ELF/VLF measurements at a ground-based receiver located near the High-Frequency Active Auroral Research Program (HAARP) HF transmitter in Gakona, Alaska, exhibit variations in si...
متن کامل